take into account the num of processes by ulimit #143614

klim-iv · 2025-07-07T21:36:02Z

No description provided.

rustbot · 2025-07-07T21:36:07Z

rustbot has assigned @tgross35.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

klim-iv · 2025-07-07T21:36:53Z

due to ulimit restrictions is not used in std::thread
apears crashes like this one:
thread 'main' panicked at .../.cargo/registry/.../rayon-core-1.12.1/src/registry.rs:168:10:
The global thread pool has not been initialized.: ThreadPoolBuildError
 { kind: IOError(Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }) }

tgross35 · 2025-07-07T21:51:46Z

library/std/src/sys/pal/unix/thread.rs

@@ -440,13 +440,24 @@ pub fn available_parallelism() -> io::Result<NonZero<usize>> {
                    }
                }
            }
+
+            let mut ulimit = u64::MAX;


The type is rlim_t, this won't work on x86-32

tgross35 · 2025-07-07T21:52:25Z

library/std/src/sys/pal/unix/thread.rs

+            #[cfg(any(target_os = "android", target_os = "linux"))]
+            {


The above block has the same cfg, this can be combined

tgross35 · 2025-07-07T21:58:54Z

Could you please put a more detailed description of how the problem is reproduced in the PR description? Or open a separate issue for it.

The change here also needs comments, and the docs at https://doc.rust-lang.org/beta/std/thread/fn.available_parallelism.html should be updated if this winds up being accepted.

Since this is a user-visible change this will need t-libs signoff, but first cc @cuviper since it looks like the issue shows up in Rayon (possibly rayon-rs/rayon#694?)

cuviper · 2025-07-07T22:18:54Z

I doubt that this will be useful -- e.g. starting a number of threads equal to the current RLIMIT_NPROC is already doomed by the existence of the current thread, nevermind any other threads or even other processes.

cuviper · 2025-07-07T22:23:44Z

library/std/src/sys/pal/unix/thread.rs

+                let mut r: libc::rlimit = unsafe { mem::zeroed() };
+                unsafe {
+                    if libc::getrlimit(libc::RLIMIT_NPROC, &mut r) == 0 {
+                        ulimit = r.rlim_max


Note, the "soft" rlim_cur is the value that is currently enforced, while the "hard" rlim_max is a limit on how high an unprivileged process can adjust its own rlim_cur.

klim-iv · 2025-07-08T10:47:18Z

I doubt that this will be useful -- e.g. starting a number of threads equal to the current RLIMIT_NPROC is already doomed by the existence of the current thread, nevermind any other threads or even other processes.

I see, patchset updated according this comment (will count working threads)

klim-iv · 2025-07-08T12:44:20Z

Could you please put a more detailed description of how the problem is reproduced in the PR description? Or open a separate issue for it.

The change here also needs comments, and the docs at https://doc.rust-lang.org/beta/std/thread/fn.available_parallelism.html should be updated if this winds up being accepted.

Since this is a user-visible change this will need libs-api signoff, but first cc @cuviper since it looks like the issue shows up in Rayon (possibly rayon-rs/rayon#694?)

bug here: rust-lang/rustup#4403

cuviper · 2025-07-08T17:55:11Z

I doubt that this will be useful -- e.g. starting a number of threads equal to the current RLIMIT_NPROC is already doomed by the existence of the current thread, nevermind any other threads or even other processes.

I see, patchset updated according this comment (will count working threads)

This makes it possible, as long as nothing else starts in the meantime, but I still don't think it's a good idea for a threadpool to completely and greedily claim the entire thread limit. (I also replied in rust-lang/rustup#4403 (comment).)

It would be better to set a limit like RAYON_NUM_THREADS that leaves sufficient headroom for the rest of your workload, across all of your processes, but that isn't something we can automatically determine from here. "Sufficient" may depend on a lot of factors, including future behavior.

Fundamentally, these resource limits are quite different than the basic "number of logical CPUs" view of available parallelism, because it's not really a problem to oversubscribe CPUs.

klim-iv · 2025-07-08T19:12:08Z

completely and greedily claim the entire thread limit

this patchset is not about 'greedily claim', here we are just return number of available parallel tasks
'greedily claim' appears in rayon, based on this info

anyway, I just want to help and show the problem, if you think that this issue is not "big deal" - ok, I just close request

cuviper · 2025-07-08T19:45:46Z

completely and greedily claim the entire thread limit

this patchset is not about 'greedily claim', here we are just return number of available parallel tasks 'greedily claim' appears in rayon, based on this info

Please understand, I did not mean any judgement in calling it "greedy".

It is the documented expectation of available_parallelism that this is a good number of threads to use, right in its headline: "Returns an estimate of the default amount of parallelism a program should use." That makes sense in general for the number of CPUs, but I would not recommend that a program should use their maximum RLIMIT_NPROC all at once.

anyway, I just want to help and show the problem, if you think that this issue is not "big deal" - ok, I just close request

I am just one voice here -- it's possible that other team members may feel differently.

due to ulimit restrictions is not used in std::thread upears crashed like this one: thread 'main' panicked at .../.cargo/registry/.../rayon-core-1.12.1/src/registry.rs:168:10: The global thread pool has not been initialized.: ThreadPoolBuildError { kind: IOError(Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }) } resolve #143635

tgross35 · 2025-07-08T20:06:06Z

I'm not sure what would be best here so
r? @cuviper

the8472 · 2025-07-09T12:58:26Z

from the rustup issue:

For hosting my web projects I am using Linux on 3-rd party cloud provider
On that system installed 104 CPUs and also set maximum number of processes for user = 40 ( by ulimit -u)

nproc counts per-user threads, so it's a shared cap. An individual process can't really account for that and available_parallelism explicitly says

does not account for differences in (co)processor capabilities or current system load

I.e. won't look at what other processes are doing.

Maybe we can do something conservative here, if something sets a ridiculously tight nproc limit (<= number of CPUs) then we just clamp parallelism to 1 because we don't know what else might be running under that UID and we don't want to eat up all the threads.

Or maybe rayon should have a limited-effort mode where it spawns threads if it can but doesn't explode when the system doesn't let it.

klim-iv · 2025-07-17T14:20:47Z

one more issue related to wrong behavior of std::thread::available_parallelism in Linux systems with ulimit restrictions:
rust-lang/rustup#4416

Amanieu · 2025-07-17T18:38:18Z

We discussed this in the @rust-lang/libs meeting. There are many reasons why thread creation may fail, and fundamentally available_parallelism cannot cover all of them. In your particular case you have a rather contrived setup with a process count limit. Rather than trying to support all the ways in which parallelism can be limited, it would be better to instead offer a generic way to limit parallelism, possibly via an environment variable. This could be proposed via an ACP.

@rfcbot close

rfcbot · 2025-07-17T18:38:20Z

Team member @Amanieu has proposed to close this. The next step is review by the rest of the tagged team members:

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

cuviper · 2025-07-17T18:54:13Z

it would be better to instead offer a generic way to limit parallelism, possibly via an environment variable.

For example, this could be similar to RUST_MIN_STACK:
https://doc.rust-lang.org/nightly/std/thread/index.html#stack-size

klim-iv · 2025-07-17T21:09:47Z

I glad to listen that you drew attention to the problem
Looks like available_parallelism should be deleted at all, or updated it description with refactor all crates that using it (IMHO):

    Returns an estimate of the default amount of parallelism a program should use.

the8472 · 2025-07-17T21:30:30Z

Looks like available_parallelism should be deleted at all

Your use-case is uncommon, that is not sufficient motivation to deprecate an existing, widely-used std API.

Software looking at cpumasks, cgroups and number of processor cores and then returning the minimum of that is common practice. Java, C#, C++ all do some of that. None of them look at nprocs.

That is because nprocs is commonly set to a much larger number, and when it is limited it is really difficult to account for because it's shared across processes. It's more like running out of disk space.... you only notice when you hit the limit.

Using nprocs to limit shared workloads feels like a technique from the 90s. It's fine if you're running a bunch bunch of php or perl scripts or whatever. It doesn't interact well with modern multi-threaded runtimes.

One can write Rust like that, but most projects don't default to few-/single-threaded behavior and you have to explicitly opt in to that via arguments or environment variables.

I think adding an override for std would be reasonable since it'll avoid having to tweak things for each crate separately.

rustbot assigned tgross35 Jul 7, 2025

rustbot added O-unix Operating system: Unix-like S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Jul 7, 2025

tgross35 reviewed Jul 7, 2025

View reviewed changes

This comment has been minimized.

Sign in to view

cuviper reviewed Jul 7, 2025

View reviewed changes

klim-iv force-pushed the master branch 2 times, most recently from be59dbd to 65b3d80 Compare July 8, 2025 12:34

klim-iv mentioned this pull request Jul 8, 2025

Exception "Resource temporarily unavailable" appears during executing rustup on Linux system with ulimit restrictions rust-lang/rustup#4403

Closed

This comment has been minimized.

Sign in to view

klim-iv force-pushed the master branch from 65b3d80 to 64b98a4 Compare July 8, 2025 19:01

This comment has been minimized.

Sign in to view

klim-iv force-pushed the master branch from 64b98a4 to 24e029f Compare July 8, 2025 19:55

rustbot assigned cuviper and unassigned tgross35 Jul 8, 2025

cuviper added the I-libs-nominated Nominated for discussion during a libs team meeting. label Jul 8, 2025

rfcbot added proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. disposition-close This PR / issue is in PFCP or FCP with a disposition to close it. labels Jul 17, 2025

Amanieu removed the I-libs-nominated Nominated for discussion during a libs team meeting. label Jul 17, 2025

take into account the num of processes by ulimit #143614

Are you sure you want to change the base?

take into account the num of processes by ulimit #143614

Conversation

klim-iv commented Jul 7, 2025

Uh oh!

rustbot commented Jul 7, 2025

Uh oh!

klim-iv commented Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tgross35 Jul 7, 2025

Choose a reason for hiding this comment

Uh oh!

klim-iv Jul 8, 2025

Choose a reason for hiding this comment

Uh oh!

tgross35 Jul 7, 2025

Choose a reason for hiding this comment

Uh oh!

klim-iv Jul 8, 2025

Choose a reason for hiding this comment

Uh oh!

tgross35 commented Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

cuviper commented Jul 7, 2025

Uh oh!

cuviper Jul 7, 2025

Choose a reason for hiding this comment

Uh oh!

klim-iv Jul 8, 2025

Choose a reason for hiding this comment

Uh oh!

klim-iv commented Jul 8, 2025

Uh oh!

klim-iv commented Jul 8, 2025

Uh oh!

This comment has been minimized.

cuviper commented Jul 8, 2025

Uh oh!

klim-iv commented Jul 8, 2025

Uh oh!

This comment has been minimized.

cuviper commented Jul 8, 2025

Uh oh!

tgross35 commented Jul 8, 2025

Uh oh!

the8472 commented Jul 9, 2025

Uh oh!

klim-iv commented Jul 17, 2025

Uh oh!

Amanieu commented Jul 17, 2025

Uh oh!

rfcbot commented Jul 17, 2025 • edited by cuviper Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cuviper commented Jul 17, 2025

Uh oh!

klim-iv commented Jul 17, 2025

Uh oh!

the8472 commented Jul 17, 2025

Uh oh!

Uh oh!

klim-iv commented Jul 7, 2025 •

edited

Loading

tgross35 commented Jul 7, 2025 •

edited

Loading

rfcbot commented Jul 17, 2025 •

edited by cuviper

Loading